# Large language model

NoteLLM is a retrieval-based large language model focused on user-generated content, aiming to enhance the performance of recommendation systems. By combining topic generation with embedding generation, NoteLLM improves its ability to understand and process note content. The model adopts an end-to-end fine-tuning strategy, supporting multi-modal inputs, which enhances its application potential in diversified content domains. Its importance lies in effectively improving the accuracy of note recommendations and user experience, especially suitable for UGC platforms like Xiaohongshu.

Baichuan-M1-14B is an open-source large language model developed by Baichuan Intelligent, optimized for medical scenarios. It is trained on high-quality medical and general data comprising 200 trillion tokens, covering over 20 medical departments, showcasing strong contextual understanding and long-sequence task performance. The model excels in healthcare applications and achieves state-of-the-art performance in general tasks for models of similar sizes. Its innovative model architecture and training approach enable outstanding performance in complex tasks such as medical reasoning and diagnosis, providing robust support for AI applications in healthcare.

EXAONE-3.5-32B-Instruct-GGUF is a series of instruction-tuned bilingual (English and Korean) generative models developed by LG AI Research, with versions ranging from 2.4B to 32B parameters. These models support long-context processing up to 32K tokens, showcasing state-of-the-art performance in real-world use cases and long-context understanding while remaining competitive in the general domain compared to other recently released models of similar scale. This model series offers detailed information through technical reports, blogs, and GitHub, and includes various precision instruction-tuned 32B language models with the following features: 30.95B parameters (excluding embeddings), 64 layers, GQA attention heads, 40 Q-heads and 8 KV-heads, a vocabulary size of 102,400, a context length of 32,768 tokens, and quantization options such as Q8_0, Q6_0, Q5_K_M, Q4_K_M, IQ4_XS in GGUF format (including BF16 weights).

lmms-finetune is a unified codebase designed to simplify the fine-tuning process for large multimodal models (LMMs). It provides a structured framework that allows users to seamlessly integrate and fine-tune cutting-edge LMMs, supporting full fine-tuning as well as strategies like LoRA. The codebase is lightweight and straightforward, making it easy to understand and modify, and it supports various models including LLaVA-1.5, Phi-3-Vision, Qwen-VL-Chat, LLaVA-NeXT-Interleave, and LLaVA-NeXT-Video.

Meta Llama 3 is the latest large language model launched by Meta, designed to unlock the capabilities of large language models for individuals, creators, researchers, and businesses. The model features different scaling versions ranging from 8B to 70B parameters and supports pre-training and instruction tuning. It is available through a GitHub repository, enabling users to perform local inference by downloading the model weights and tokenizer. The release of Meta Llama 3 signifies the further democratization and application of large language model technology, exhibiting broad research and commercial potential.

Mistral-Nemo-Instruct-2407 is a large language model (LLM) jointly trained by Mistral AI and NVIDIA, which is an instruction-tuned version of Mistral-Nemo-Base-2407. The model has been trained on multilingual and code data and has significantly outperformed existing models of similar or smaller size. Its main features include: supporting multilingual and code data training, 128k context window, and can be replaced with Mistral 7B. The model architecture includes 40 layers, 5120 dimension, 128 head dimension, 1436 hidden dimension, 32 heads, 8 kv heads (GQA), 2^17 vocabulary (about 128K), rotor embedding (theta=1M). The model has performed well on various benchmarks, such as HellaSwag (0-shot), Winogrande (0-shot), OpenBookQA (0-shot) etc.

Skywork-MoE is a high-performance Mixture of Experts (MoE) model with 14.6 billion parameters, consisting of 16 experts and 2.2 billion activation parameters. This model is initialized from the dense checkpoint of the Skywork-13B model and incorporates two innovative techniques: gated logit normalization to enhance expert diversification, and adaptive auxiliary loss coefficients allowing for layer-specific auxiliary loss coefficient adjustment. Skywork-MoE demonstrates comparable or better performance than models with more parameters or activation parameters, such as Grok-1, DBRX, Mistral 8*22, and Deepseek-V2.

MoMA Personalization is a personalized image generation tool based on an open-source Multimodal Large Language Model (MLLM). It focuses on theme-driven personalized image generation, capable of generating high-quality images that preserve the target object features based on reference images and text prompts. MoMA requires no fine-tuning and acts as a plugin model, directly applicable to existing diffusion models. It enhances the detail and prompt fidelity of generated images while maintaining the original model's performance.

Dust is an AI assistant that securely delivers the best large language models to your team, along with continually updated company knowledge, powerful collaboration applications, and a scalable platform. Using Dust, you can access GPT-4 in a unified and secure way, receiving consistently updated answers in a chat interface to help your team better understand its environment and make more informed decisions. Additionally, Dust allows your team to share prompts and conversations, quickly discovering the potential of generative AI in their tasks. Based on ongoing internal dialogue and decisions, Dust can offer suggestions for document updates and improvements. You can also build your own powerful workflows on Dust, creating custom large language model applications using company data and letting Dust handle the complexities of LLM implementation to adapt to specific needs.